Depth Observation for Image - based Visual Servoing : Theory

In the classical image-based visual servoing framework, error signals are directly computed from image feature parameters, allowing, in principle, control schemes to be obtained that need neither a complete three-dimensional (3D) model of the scene nor a perfect camera calibration. However, when the computation of control signals involves the interaction matrix, the current value of some 3D parameters is required for each considered feature, and typically a rough approximation of this value is used. With reference to the case of a point feature, for which the relevant 3D parameter is the depth Z, we propose a visual servoing approach where Z is observed and made available for servoing. This is achieved by interpreting depth as an unmeasurable state with known dynamics, and by building a non-linear observer that asymptotically recovers the actual value of Z for the selected feature. A byproduct of our analysis is the rigorous characterization of camera motions that actually allow such observation. Moreover, in the case of a partially uncalibrated camera, it is possible to exploit complementary camera motions in order to preliminarily estimate the focal length without knowing Z. Simulations and experimental results are presented for a mobile robot with an on-board camera in order to illustrate the benefits of integrating the depth observation within classical visual servoing schemes. KEY WORDS—Image-Based Visual Servoing, Nonlinear Observers, Depth Observation, Focal Length Observation, Mobile Robots The International Journal of Robotics Research Vol. 27, No. 10, October 2008, pp. 1093–1116 DOI: 10.1177/0278364908096706 c SAGE Publications 2008 Los Angeles, London, New Delhi and Singapore Figures 2–6, 8–18 appear in color online: http://ijr.sagepub.com